Augmented Reality (AR) includes the provision of virtual objects over a view of the real-world environment to provide operational information, gameplay experiences, and information overlays to users. Various devices may be used to provide AR experiences to users. Some of these AR devices project virtual objects onto a reflecting medium to overlay virtual objects onto a real-world environment visible to a user. Other examples of AR devices provide a video feed into which the virtual objects are inserted for display to the user. In each case, the AR device may determine an orientation of the Field of View (FOV) for the user in the environment and identify various real objects, so that virtual objects may be displayed in consistent and appropriate locations for the intended AR experience.